A wide range of sensor technologies (visible, infrared, radar, etc.) and a wide variety of platforms (mountaintops, aircraft, satellite, etc) are currently used to obtain imagery of planetary surfaces. Geo-registration is the process of mapping imagery obtained from such various sources into predetermined planetary coordinates and conditions, i.e. calibrating/correlating such image data to the real world so as to enable, for example, the determination of absolute positions (e.g. GPS (global positioning system) coordinates), distances, etc. of features found in the image. However, in order to overlay the images from the various types of cameras (sensor fusion), the image data from all of the disparate sources must first be modified into a common coordinate system. Geo-rectification is the process of converting or otherwise transforming an image (e.g. an off-axis image) recorded from an arbitrary position and camera orientation, into one that appears as a nadir view, i.e. a view from directly above the scene/object/features of interest looking straight down, as at a map. Geo-rectification thus enables various images to share the same orthogonal perspective so that they can be geo-registered and correlated to/against each other or a reference image.
Image processing, however, is often computationally expensive; the number of image processing computations necessary to perform geo-rectification of off-axis high-resolution images is typically very large even for small source images, requiring significant computational resources and making real-time visualization of live data difficult. Real-time image data processing is therefore typically managed as a trade off between image size (number of pixels) and data rate (frames per second).
Current techniques for fast image rendering and geo-registration either employ software only for post processing of data, or require expensive custom hardware i.e. dedicated pixel processors, even for relatively low-resolution source data. Software-only techniques, for example, can perform the image transformation calculations necessary for geo-registration on the central processing unit (CPU) of a computer or workstation. Due to inadequate memory bandwidth, however, these methods typically take 2-3 seconds per mega-pixel of image data, even with currently available high-end workstations preventing such software only methods from performing in real-time.
And the custom hardware approach typically utilizes dedicated pixel processors, which are specially designed graphics cards (printed circuit boards) and software capable of high throughputs which enable real-time performance of image transformations and geo-registration. For example, one particular custom hardware/dedicated pixel processor for performing real time geo-registration, known as Acadia™, is commercially available from Sarnoff/Pyramid Vision Technologies. This representative custom device, however, operates at a low resolution with RS-170 quality video, which is ˜640×480 pixels at 30 Hz Moreover, there is a high cost for such custom hardware and the programming time to custom-configure such hardware. For example, such custom dedicated pixel processors typically cost in the tens of thousands of dollars for the hardware alone, and an additional cost ranging up to $100K for the configuration of the software.
What is needed therefore is a digital image processing methodology for performing geo-registration that is faster (real time streaming), more cost effective, and with higher resolution than software-only techniques or the use of expensive custom hardware/dedicated pixel processors.